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PREFACE 
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The  present  report  is  the  last  in  a  series  of  three  reports  ’  which 
describe  the  inner  workings  of  MANMODEL--a  name  which  has  become  associated 
with  our  attempt  to  produce  a  simulator  which  would  yield  measures  of 
system  perfoi'mance  under  different  mixes  of  equipment,  personnel,  and 
procedures.  The  work  lias  not  come  to  an  end.  Rather,  a  mid-point  mile¬ 
stone  has  been  reached  with  the  issuing  of  this  report.  Consequently,  it 
may  be  of  interest  to  trace  the  history  of  MANMODEL's  development.  This 
brief  preface  will  not  be  the  whole  story  but  hopefully  will  disclose 
the  lessons  learned,  in  order  that  others  can  capitalize  on  the  good  points 
and  avoid  the  pitfalls. 

MANMODEL  began  as  a  by-product  from  ARI ' s  Field  Unit  approach  toward 
bridging  the  gap  between  the  laboratory  and  the  field.  The  Field  Unit 
approacli  may  be  described  as  follows:  you  selectively  structure  a  human 
factors  group  of  personnel  who  are  primarily  laboratory  oriented,  place 
them  on-site  as  an  integral  part  of  the  design  and  evaluation  process,  and 
in  this  manner  provide  a  two-way  transducer  between  research  and  applica¬ 
tions.  For  this  project  an  ARI  Field  Unit  was  formed  and  co-located  in 
Heidelberg,  Germany  with  personnel  from  the  U.S.  Army  Computer  Systems 
Command,  Headquarters  U.S.  Army  Europe  and  Seventh  Army  to  provide  across- 
the-board  human  factors  support  while  the  U.S.  Army's  first  automated 
tactical  operations  system  (TOS)  was  being  evolved  in  the  hands  and  the 
setting  of  the  users. 

This  phase  of  what  is  still  an  ongoing  TOS  effort  took  place  in  1967- 
1970.  During  that  period  considerable  human  factors  research  was  conducted, 
both  in  the  laboratory  and  the  field,  to  answer  the  many  human  factors 
problems  as  they  arose.  3’  4  Before  long,  it  became  evident  that  a  framework 


1  Siegel,  A.  I.,  J.  J.  Wolf,  and  W.  R.  Leahy.  A  digital  simulation  model  of 
message  handling  in  the  tactical  operations  system:  I.  The  model,  its 
sensitivity,  and  user's  manual,  ARI  Research  Memorandum  73~5>  November 

1973. 

2  Siegel,  A.  I,.  J.  J.  Wolf,  W.  R«  Leahy,  and  J.  L.  Bearde.  A  digital 
simulation  model  of  message  handling  in  the  tactical  operations  system: 

II.  Extensions  of  the  model  for  interactivity  with  subjects  and 
experimeters.  ARI  Research  Memorandum  73~S,  December  1973* 

3  Ringel,  S.,  J.  D.  Baker,  M.  H.  Strub,  and  I,.  L.  Kensinger.  Human  Factors 
research  in  command  information  processing  systems--summary  of  recent 
studies.  ARI  Research  Study  G9-6,  May  1 9<?9* 

4  Baker,  J.D.  Acorns  in  flower  pots/psychologists  in  the  field.  JSAS 
Catalog  of  Selected  Documents  in  Psychology,  1972,  2,  p.88. 
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was  needed  which  would  link  the  implications  from  the  human-performance 
data  being  generated  to  system  performance.  Additionally,  TOS  was  rapidly 
reaching  a  point  where  the  design  verification  evaluation  efforts  would 
require  the  development  of  a  cohesive  test  plan  to  integrate  all  aspects 
and  relationships  of  the  man-machine  interface  and  tie  them  to  the  system 
performance  measures.  From  these  efforts  the  form  of  the  present  model 
began  to  emerge. 

An  on-site  visit  from  Professor  Bernard  Metz  of  the  Centre  D'Etudes 
Bioclimatique  in  Strasbourg,  France,  provided  the  impetus  to  add  substance 
to  the  form.  In  describing  our  work  and  how  each  aspect  interrelated,  a 
skeleton  of  the  model  was  used  as  a  framework  for  presentation.  Professor 
Metz  became  very  much  interested  in  the  concept  itself  and  suggested  that 
the  overall  idea  be  further  articulated,  key  points  elaborated  upon,  and 
the  results  presented  at  the  Fourth  International  Congress  on  Ergonomics. 
The  outcome  was  the  birth  of  MANMODEL  as  a  model  .5 

But  to  be  valuable  as  a  general-purpose  tool  in  support  of  ongoing  and 
future  Army  tactical  data  system  efforts,  the  model--this  representation 
of  reality--needed  to  be  further  developed  into  a  simulator.  This  required 
that  complex  logic,  structure,  and  software  be  developed  to  produce  a 
simulation  vehicle  which  would  combine  the  effects  of  such  variables  as 
message  queuing,  detailed  message  processing  procedures,  error  rates, 
and  personnel  characteristics--along  with  stochastic  variations--to 
yield  predictions  of  system  performance.  It  was  decided  that  the  work 
would  be  accomplished  through  contract  support;  thus  began  this 
professionally  satisfying  and  highly  productive  association  with  Dr. 

Arthur  Siegel  and  his  colleagues. 

The  first  version  of  the  MANMODEL  simulator  required  card-punch  input, 
used  batch  processing,  and  produced  hard-copy  line-printer  output.  During 
the  development  of  this  version  of  the  simulator  we  encountered  one  of 
our  lessons--if  you  can  aggregate  items  without  significant  loss  of  predic¬ 
tive  power,  do  so,  for  it  will  make  your  model  both  general  and  more 
manageable.  The  case  in  point  here  was  that  the  original  error  schema 
developed  from  MANMODEL  °  was  found  to  be  much  too  specific  to  a  particular 
system  which  in  itself  had  some  peculiar  characteristics.  To  offset  this 
weakness,  a  more  general  error  schema  was  devised.6 7 


6  Baker,  J.D.  Quantitative  modeling  of  human  performance  in  information 
systems,  Ergonomics ,  1970*  _15  (6),  645-664.  ARI  Technical  Paper  252, 

(AD  746-096). 

6  Baker,  1970,  op.  cit. 

7  Nawrocki,  L.  H. ,  M.  H.  Strub,  and  R.  M.  Cecil.  Error  categorization  and 
analysis  in  man-computer  communication  systems,  IEEE  Transactions  on 
Reliability ,  Vol.  R-22,  August  1975* 
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When  Version  I  became  operational  it  was"  realized  that  MANMODEL  would 
be  more  useful  if  a  designer  or  researcher  were  able  to  sit  on-line  and 
pose  "what  if"  questions.  Further,  the  "what  if"  questions  could  be 
successively  refined  if  the  results  of  each  run  were  rapidly  summarized 
and  displayed  for  the  user.  This  modification  produced  Version  II  in 
which:  (l)  appropriate  parameters  would  be  displayed  on  a  CRT  for  the 

per soil  using  the  model;  (2)  he  could  manipulate  these  parameters  on-line; 
(p)  he  would  immediately  see  displayed  a  summary  of  the  results  of  this 
change  on  system  performance;  and  (4)  hard-copy  printouts  of  the  detailed 
interactions  could  subsequently  be  acquired. 

At  this  stage  of  the  development  a  further  extension  of  the  model 
appeared  to  be  possible. 

A  ...  benefit  inherent  in  this  approach  is  that  it  is 
modular  and  permits  plugging  human  performance  studies 
directly  into  a  system  framework  ...  (One  technique) 
would  be  to  start  up  the  model  with  randomly  sampled 
values  for  the  parameters  for  each  node  in  the  data 
flow  and  processing  dimension  ...  Into  this  flow 
we  introduce  an  on-line  human  factors  study  which  may 
have  as  its  basic  goal  the  development  of  performance 
measures  along  the  task  analysis  dimension,  but  which 
could  be  providing  simultaneous  measures  of  the 
perturbations  the  human  is  making  on  system  output. 

With  the  knowledge  gained  in  getting  Version  II  of  MANMODEL  operating, 
the  above  idea  did  not  seem  too  far-  fetched.  Thus,  yet  a  third  version 
of  the  model  was  undertaken.  The  result  was  the  production  of  a  man-in- 
thc-loop  hybrid  simulation  vehicle  which  permits  the  MANMODEL  program  to 
operate  in  the  background  while  being  simultaneously  responsive  to  two 
on-line  sources.  One  source  is  the  experimental  subject  who  is  providing 
his  own  performance  data  as  inputs;  the  second  is  an  on-line  experimenter/ 
monitor  who  is  providing  as  input  information  concerning  those  activities 
of  the  subject  which  the  computer  is  incapable  of  sensing.  This  third 
version  of  MANMODEL  is  the  topic  of  the  present  report. 

The  reader  should  be  cautioned  that  correspondence  is  not  exact  between 
the  three  versions  of  MANMODEL  and  the  three  reports  describing  it.  For 
example,  portions  of  Version  III  arc  described  in  Volume  II  of  this  series. 


Baker,  1970,  op.  cit.,  P.  662. 
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The  versions  coincide  with  the  events  that  gave  rise  to  their  conceptual¬ 
ization;  the  volumes  arc  based  on  logical  developmental  milestones 
war  ranting  documentation. 


Continued  work  is  planned  in  using  and  improving  this  model.  One 
major  activity  which  MANMODEL  will  support  is  the  cost-effectiveness 
analysis  efforts  associated  with  the  current  TOS  development.  To  this 
end  MANMODEL,  which  now  runs  in  22K  core  of  memory  on  the  ARI  CDC  3300 
computer  facility,  will  be  installed  on  a  UNIVAC  110,.  system  to  which 
ARI  has  time-shared  access.  The  additional  memory  provided  by  the  1108 
will  allow  for  experimentation  toward  interfacing  MANMODEL  with  two 
other  models  being  used  in  support  of  the  TOS  cost-effectiveness  analysis-- 
CASE  and  TOS/SAM.  CASE  simulates  the  flow  under  realistic  conditions  of 
multichannel  communications  traffic  through  a  network  of  links  and  nodes. 
TOS/SAM  simulates  the  hardware  and  software  of  the  computer,  input/output 
devices,  and  the  communications  links  within  the  TOS. 

While  some  preliminary  tests  of  the  validity  of  MANMODEL  have  been 
conducted,  with  satisfying  results,  further  work  toward  validating  this 
simulator  is  both  warranted  and  planned. 

Possibly  more  important  for  scientific  psychology  are  the  implica¬ 
tions  for  research  from  lessons  learned  during  the  development  of  Version 
III.  To  elaborate,  when  MANMODEL  is  being  run  as  a  pure  computer 
simulation  (e.g.,  as  Version  I  or  II)  handling  such  sources  of  variation 
as  motivation,  level  of  aspiration,  and  stress  is  not  too  difficiilt. 

The  motivational  algorithm,  for  example,  is  theoretically  sound  and  the 
outputs  behave  in  a  reasonable  and  expected  way. 

However,  while  we  were  developing  Version  III  of  MANMODEL  (the  man- 
in-the-loop  hybrid  simulation  version)  we  were  visited  by  Mr.  Brian 
Venncr  of  the  Army  Personnel  Research  Establishment,  Farnborough  Hants, 
England,  lie  posed  a  simple,  yet  penetrating,  question:  How  do  you 
determine  appropriate  values  for  stress  tolerance  or  aspiration  level 
for  a  subject  who  is  about  to  be  plugged  into  the  MANMODEL  simulation 
loop?  To  date  wo  do  not  have  a  good  answer  to  that  question.  In  a 
broader  sense  the  question  could  be  addressed  to  psychology  as  a  whole. 

How  do  you  integrate  knowledge  collected  independently  and  in  the  abstract 
when  the  outcome  of  importance  really  lies  in  the  interactive  effects? 

We  all  have  heard  people  speak  of  "the  high  expirit  de  corps  of  this  unit," 
or  "the  motivation  and  morale  of  that  shift  is  low."  We  all  know  and 
sense  it,  but  how  do  we  measure  it?  How  does  one  derive  a  number  to 
represent  the  motivation  for  a  given  subject  participating  in  our  hybrid 
simulation  run?  Given  these  numbers,  what  do  they  mean?  We  have  discovered 
that  little  data  exist  which  bridge  the  gap  between  motivation  as  a 
psychological  concept  and  concrete  implications  for  human  performance  in 
a  particular  setting. 
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It  should  be  noted  that  computerised  modelling  efforts  inherently 
possess  several  characteristics  relevant  to  these  questions.  First, 
they  force  one  to  make  the  relationships  among  variables  explicit  since 
computers  cannot  deal  with  'taaybe"  statements.  Second,  because  they 
operate  on  a  GIGO  principle--garbage  in/garbage  out--data  deficiencies 
require  immediate  and  detailed  attention.  Therefore,  the  real  value  of 
MANMODEL  may  go  beyond  its  potential  contributions  to  the  TOS  development 
effort.  It  may  force  us  to  tackle  some  of  psychology's  "fuzzy"  questions 
and,  in  so  doing,  may  lead  us  to  some  interesting  answers.  Only  time 
and  further  research  will  tell  whether  we  will  be  successful  in  meeting 
this  challenge. 

(k><i  k 

Z_JjAMES  D.  BAKER 
/7  Supervisory  Project  Director 
//  Command  Systems 
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CHAPTER  I 
INTRODUCTION 


Prior  reports  of  the  series  have  described  the  logic  and  implemen¬ 
tation  of  a  digital  simulation  model  for  simulating  the  acts  and  behaviors  of 
the  crew  members  in  the  Tactical  Operations  System  (TOS).  The  first  re¬ 
port  (Siegel  ct  al. ,  1973a)  presented  a  description  of  the  stochastic  model 
and  the  results  of  initial  sensitivity  testing  and  verification.  The  second 
report  (Siegel  et  al.  ,  1973b)  described  extensions  of  the  original  model  to 
allow  interactivity  of  the  digital  computer  based  model  with  on  line  subjects 
and  with  an  experimenter.  The  present  report  describes  further  extensions 
which  increase  its  fidelity  and  utility.  The  first  extension  will  allow  the  ex¬ 
perimenter  to  describe  the  subject's  actions  not  only  by  recording  task  re¬ 
lated  behaviors,  but  also  by  identifying  and  recording  up  to  nine  task  related 
actions  classified  as  different  types  of  interruptions.  These  interruptions 
are  not  only  identified  but  are  also  timed.  Task  element  completion  is  also 
identified  and  recorded  by  the  experimenter.  These  model  extensions  re¬ 
sult  in  one  additional  step  towards  the  fidelity  of  simulation  based  on  accu¬ 
rate  task  analytic  data  as  discussed  by  Baker  (1970b). 

The  second  modification  allows  automatic  incorporation  of  data  col¬ 
lected  on  line  into  the  batch  processing  model.  Customary  card  supplied, 
model  input  data  along  with  interrupt  data  are  modified/ replaced  by  these 
new  data  according  to  specified  rules.  All  other  interactive  capabilities 
of  the  model  (for  example,  task  allocation)  are  retained. 


Overview  of  Extensions 

Figure  1  presents  an  overall  view  of  the  new  input/ output  and  proc¬ 
essing  relationships  between  the  hybrid  (interactive)  model  (Siegel  et  al.  , 
1973b)  and  the  automatic  (noninteractive)  model  (Siegel  et  al.  ,  1973a)  as 
extended.  The  hybrid  model  drives  the  CRT  displays  of  the  subject  and  ex¬ 
perimenter,  interprets  and  shuttles  messages  from  the  experimenter  to 
the  subject,  and  records  subject  responses.  Input  data  from  disk  or  tape, 
as  desired,  arc  provided  to  the  hybrid  model.  Two  sets  of  output  data  are 
provided.  One  is  the  standard  output,  as  described  in  Siegel  et  al.  ,  (1973b), 
which  is  recorded  on  disk  and  can  be  run  in  a  batch  mode  to  yield  printed 
output.  The  second  is  the  subject  performance  data  which  are  recorded  on 
a  disk  or  tape  file.  This  includes  interrupt  related  data  and  task  perform¬ 
ance  time  data  which  are  then  submitted  in  a  batch  processing  mode  to  the 
automatic  model.  Standard  output  from  the  automatic  model  as  described 
by  Siegel  et  al.  (1973a)  are  also  provided  by  this  program. 


*  j*>**/» 


Figure  1.  Functional  logic. 
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In  an  operational  environment,  task  performance  can  be  expected 
to  be  interrupted  by  either  operator  (internally)  induced  or  externally  in¬ 
duced  extraneous  activities.  These  are  the  types  of  activities  that  are 
usually  included  under  the  general  rubric  of  "miscellaneous"  in  most  task 
analyses,  battle  effort  is  usually  made  to  determine  their  properties  and 
characteristics.  The  primary  reason  usually  suggested  for  treating  these 
activities  in  such  a  manner  is  that  they  are  not  an  element  of  the  actual 
task.  Moreover,  they  are  typically  considered  "insignificant"  and /  or  un¬ 
reliable. 

Baker  (1970b)  has  provided  a  framework  for  task  performance  track¬ 
ing  in  which  the  concept  of  an  interrupt  in  an  information  system  can  easily 
be  incorporated.  His  approach  was  to  postulate  five  "basic  and  critical" 
operations  that  men  perform  in  a  message  handling  system.  These  are 
screen,  transform,  input,  assimilate,  and  decide.  The  operations  are 
viewed  as  interrelated  along  three  dimensions:  (1)  data  flow  and  processing, 
(2)  task  analysis  for  each  event  in  the  data  flow  sequence,  and  (3)  source  of 
variation  such  as  level  of  training.  The  first  dimension  includes  the  notion 
of  "flowcharting"  the  sequence  of  events  and/or  operations  such  that  start 
and  end  points  and  times  are  identified.  The  second  dimension  investigates 
".  .  .  task-equipment  interactions  that  constitute  that  portion  of  the  operator's 
job,  "  while  the  third  dimension  refers  to  sources  of  variation  external  to 
the  actual  message  handling  system.  The  identification  and  timing  of  in¬ 
terrupts  clearly  falls  within  this  model. 


D  c  v  e  1  o  p  men  t  a  1  P  roce  (1  u  r  e 

Discussions  were  held  with  Army  Research  Institute  personnel  who 
possess  experience  in  the  operations  of  the  Tactical  Operations  System  to  de¬ 
termine  the  types  of  interruptions  an  action  officer  (AO)  or  input-output  de¬ 
vice  (IOD)  operator  might  typically  encounter  in  the  performance  of  their 
respective  tasks.  The  interrupt  types  were  also  categorized  by  these  same 
personnel  into  five  distinct  (and,  to  the  extent  possible,  independent  and  mu¬ 
tually  exclusive)  groups.  These  were: 
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•  intermittent  reference  interruption 

•  accidental  interruption 

•  interpersonnel  interruption 

•  personal  comfort  interruption 

•  other  interruptions 


Intermittent  reference  was  defined  as  job -related  information  seeking 
such  as  referencing  glossaries  or  accessory  materials  or  checks  with  other 
personnel  for  specific  message  related  information.  Accidental  interrup¬ 
tion  was  defined  as  disruption  due  to  unforeseen  ,  nonpersonnel  related  events. 
This  category  of  interruption  includes  such  items  as  equipment  failure  and  re¬ 
sponses  to  emergencies. 

The  third  category,  interpersonnel  interruption,  includes  disruptions 
due  to  person-to-person  communication.  This  category  of  interruption  is 
exemplified  by  telephone  or  other  types  of  conversations  which  are  not  job- 
related.  Personal  comfort  includes  interruptions  for  the  operator's  personal 
convenience,  such  as  an  unscheduled  coffee  break. 

The  final  category  is  intended  to  include  any  interruptions  not  easily 
placed  in  any  of  the  four  identified  categories. 

In  addition  to  the  development  of  categories  of  interruptions,  fre¬ 
quency  and  range  (for  time)  estimates  were  also  obtained  from  these  per¬ 
sonnel  for  each  interrupt  category. 


New  Model  Logie  and  Displays 

(This  section  replaces  p.  35  (all  text  after  Figure  3-14)  and  p.  36  (un¬ 
til  the  paragraph  beginning  "The  display.  .  .  ")  in  Siegel  et  al.  (1973b). 
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The  display  shown  in  Figure  2  is  presented  to  the  experimenter 
when  the  subject  is  to  perform  a  task  element.  The  model  will  initiate 
the  timing  of  subject  performance  of  the  task  element  when  the  experi¬ 
menter  depresses  the  SEND  key. 


READY  FOR  TASK  ELEMENT  XX  OF  TASK  ANALYSIS  X 

AT  XXXX 

DEPRESS  SEND  KEY  WHEN  SUBJECT  IS  READY 


Figure  2.  Display  to  experimenter  at  the  beginning  of 
normal  task  element  performance. 


The  display  shown  in  Figure  3  is  presented  to  the  experimenter 
while  the  subject  is  performing  a  task  element.  If  the  experimenter  ob¬ 
serves  the  start  of  an  interruption  to  subject  performance,  the  experi¬ 
menter  enters  the  proper  interrupt  code  into  the  display  and  depresses 
the  SEND  key. 


O  -  INTERRUPT  CODE  (1-9) 

DEPRESS  SEND  KEY  WHEN  SUBJECT  COMPLETES 
TASK  PERFORMANCE 


-  OR  - 


ENTER  PROPER  INTERRUPT  CODE  IN  POSITION  TW'O 
AND  DEPRESS  SEND  KEY  IF  SUBJECT  IS  INTERRUPTED 
DURING  TASK  PERFORMANCE 


Figure  3.  Display  for  indicating  interruption  of  task  completion. 
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The  display  shown  in  Figure  4  is  presented  to  the  experimenter's 
CUT  immediately  after  he  has  indicated  a  subject  interruption.  The  experi¬ 
menter  may  enter  a  twenty-character  description  of  the  interruption.  He 
depresses  the  SEND  key  when  he  observes  the  completion  of  the  interruption. 


_ INTERRUPT  CODE 

ENTER  DESCRIPTION  BETWEEN  /  /  IF  DESIRED 

/  / 

DEPRESS  SEND  KEY  WHEN  INTERRUPTION  IS  COMPLETED 


Figure  4.  Interruption  description  display. 


In  the  case  of  task  element  type  2,  the  subject  is  required  to  com¬ 
plete  a  CRT  displayed  format  blank.  A  new  display,  as  shown  in  Figure  5, 
portrays  the  resulting  presentation.  In  this  task  type,  the  experimenter  is 
not  allowed  to  "end"  the  task  element.  Only  the  subject,  having  completed 
filling  out  the  CRT  provided  format,  can  end  the  task  element  by  hitting  the 
SEND  key. 


O  -  INTERRUPT  CODE  (1-9) 

ENTER  PROPER  INTERRUPT  CODE  AND  DEPRESS  SEND  KEY 
IF  SUBJECT  IS  INTERRUPTED  DURING  TASK  PERFORMANCE 


Figure  5.  Display  for  indicating  interruption  while  subject  is 
filling  out  a  CRT  generated  format. 
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Cl  I A  PTE  11  II 


IN  TEG  HAT  I  ON  OP  INTEIlllUPT  DATA  INTO  AUTOMATIC’  MODI:!, 


As  noted  in  Chapter  I,  a  record  was  written  concerning  each  experi¬ 
mental  task  performance  and  interruption  which  was  timed  in  the  hybrid 
mode.  These  data,  along  with  card  input,  are  then  submitted  as  a  batch 
processing  job  to  the  model  in  the  automatic  mode.  This  chapter  details 
the  mathematical  operations  integrating  the  required  new  card  input  data 
and  task  performance  and  interrupt  data(generated  in  the  hybrid  mode)  with 
the  automatic  model.  Input/ output  already  considered  in  the  first  two  reports 
in  this  series  (noted  as  "standard"  in  Figure  1)  will  not  be  reviewed. 

Figure  6  shows  a  sample  recording  of  task  analytic  data  which  are 
provided  as  card  input.  This  figure  corresponds  to  Figure  2-3  in  Siegel 
et  al.  (1973b)  with  the  addition  of  data  relating  to  interrupts.  Figure  6 
identifies  which  interruption  types  have  nonzero  a  priori  (preset)  probabil¬ 
ities  of  occurring  during  specific  task  elements.  That  is,  the  analyst  iden¬ 
tified  certain  interrupt  types  as  having  a  nonzero  probabilities  of  occurrence 
during  specific  task  elements  and  for  those  he  also  provided  mean  duration 
and  standard  deviation  (in  seconds)  estimates. 

Figure  7  shows  the  results  of  the  experimenter's  recording  of  addi¬ 
tional  interrupts  which  occurred  duiing  a  hybrid  simulation  and  identifies 
when  and  what  types  of  interruptions  occurred.  The  interrupt  sequence 
and  the  duration  of  each  interruption  is  noted,  along  with  the  description 
provided  by  the  experimenter  while  he  was  observing  the  subject  at  the  time 
that  the  interruption  occurred. 

Figure  8  shows  collected  data  for  task  performance  (exclusive  of  in¬ 
terrupt  time)  by  task  element.  The  heading  "order"  indicates  the  order  in 
which  the  data  were  collected. 

Figure  9  shows  the  mean  duration  of  each  type  of  interrupt  for  each 
task  analytic  element  for  task  analysis  1.  Similar  data  are  provided  for  each 
task  analysis.  Entries  represent  mean  weighted  interrupt  times  and  are  em¬ 
ployed  as  input  to  the  automatic  model.  They  are  calculated  for  each  inter¬ 
rupt  as  follows: 

If  ten  or  fewer  interrupts  of  a  given  type  were  observed  during 

data  collection  (within  each  task  analysis  and  for  each  task  element) 


Weighted  Mean 
Interrupt  Time 


[  (Total  interrupt  time)  +  (Preset  mean  interrupt  time) 
x  (10  -  Number  of  interrupts)]  /  10 
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Figure  6.  Sample  input  task  analytic  data. 
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Figure  7.  Sample  interrupt  data. 
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Figure  !).  Sample  integration  of  interrupt  time  data. 
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If  more  than  ten  interrupts  of  that  type  were  observed  within  a 
given  task  analysis  and  for  each  task  element,  then  the  mean  data  from 
data  collection  completely  replaces  the  preset  values. 

Calculations  for  probability  of  occurrence  of  each  interrupt  type/ 
task  analysis/ task  element  are  similarly  computed  and  printed  in  tables 
such  as  shown  in  Figure  10.  These  (new)  probabilities  are  also  provided 
as  input  to  the  automatic  model.  Thus,  if  sufficient  data  have  been  col¬ 
lected,  preset  values  are  completely  ignored.  If  insufficient  data  are 
available,  a  weighting  scheme  is  employed  where  the  impact  of  collected 
data  on  the  final  values  employed  are  a  function  of  their  stability. 

Figure  11  presents  a  sample  of  the  task  element  means  and  stand 
ard  deviations  (repeated  for  each  task  analysis)  provided  as  input  to  the 
automatic  model.  Preset  values  are  employed  unless  data  have  been  ob¬ 
tained  using  the  hybrid  mode.  Then,  the  mean  of  the  collected  data  is 
substituted  for  the  preset  values  giving  new  values  to  the  task  elements 
of  the  task  analyses  which  reflect  subject  data. 

Figure  12  parallels  Figure  2-5  in  Siegel  et  al. ,  1573b,  with  the 
addition  of  interrupt  durations  and  types.  The  interrupt  times,  together 
with  the  task  performance  times,  are  included  in  the  cumulative  time. 
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Figure  10.  Sample  integration  of  interrupt  probability  data. 
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Figure  11.  Sample  task  element  data. 


CHAPTER  III  • 


DISCUSSION 


The  inclusion  of  data  collection  concerning  interrupts  represents  a 
step  in  increasing  the  fidelity  of  the  computer  simulation  of  operator  per¬ 
formance  in  the  TOS  system.  Too  often  "extraneous1  events  (which  we  have 
categorized  here  as  interrupts)  are. ignored  witn  little  effort  being  made  at 
their  identification  and  quantification.  As  such,  any  predictable  regularities 
with  respect  to  their  type  and  magnitude  are  lost  leading  to  a  greater  degree 
of  unpredictable  variance  than  might  otherwise  be  present  when  simulating 
task  performance.  The  simple  fact  that  categories  for  interruptions,  in¬ 
dependent  of  actual  task  elements,  were  developed  without  great  difficulty 
suggests  that  some  regularities  in  these  "extraneous"  events  can  be  quan¬ 
tified. 

The  second  major  addition  to  what  will  be  referred  to  henceforth  as 
the  MANMODEL  is  the  ability  to  collect  data  in  an  interactive  mode  using 
any  combination  of  real  and  simulated  subjects  and  then  immediately  simu¬ 
late  (in  an  automatic  mode)  the  entire  interactive  process.  An  experimenter- 
can  thus  employ  real  subjects  for  data  collection  and  then  almost  immedi¬ 
ately  simulate  the  entire  message  handling  procedure.  This  procedure  al¬ 
lows  him  the  ability  to  manipulate  variables  during  data  collection  and  then 
by  using  the  automatic  mode  of  the  MANMODEL  to  measure  effects  using 
this  "new"  message  handling  procedure.  Different  system  configurations 
based  on  actual  data  can  thus  be  manipulated  with  their  effects  becoming 
almost  immediately  apparent. 

As  sensitivity  analyses  have  already  been  completed,  a  logical  next 
step  in  the  development  of  the  MANMODEL  would  be  a  validation  of  the 
model's  predictions.  Full  confidence  cannot  be  placed  in  the  techniques 
and  logic  employed  in  the  model  until  one  or  more  empirical  validations 
have  taken  place.  These  validations  might  take  the  form  of  having  teams 
of  operators  perform  for  a  specified  period  in  a  TOS-type  message  handling 
system.  Alternative  teams  would  experience  different  types  of  messages  or 
the  same  types  under  different  types  of  circumstances  (for  example,  more 
interrupts).  Model  predictions  might  then  be  validated  against  obtained  data 
possibly  by  using  the  statistical  procedure  developed  by  Siegel  et  al.  (1972). 


»w.i-  Jm 
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APPENDIX  A 


Additions  and  Modifications  to  the  Interactive  TOS  Model  for 

Interrupt  Consideration 


The  modifications  noted  below  should  be  made  in  the  report: 
A  Digital  Simulation  Model  of  Message  Handling  in  the  Tactical  Operations 
System:  II.  Extensions  of  the  Model  for  Interactivity  with  Subjects  and 
Experimenters  by  Arthur  I.  Siegel,  J.J.  Wolf,  W.R.  Leahy,  and 
J.L.  Pearde.  References  to  pages  deleted  will  refer  to  that 
report . 


Control  Cards  for  Automatic  Model 

1 .  Sequence  card  (this  card  and  all  others  here  listed  have  a 
7  and  a  9  in  column  1 

2 .  Job  card 

3.  FET, CECIL, OVERLAYS, , , 

4.  MODIFY , , ,NEWFET 

5.  FET, CECIL, OVERLAYS, 51 2, , , 

6.  OPEN, 25 

7 .  FET ,TEMP , SCRATCH  1 , , , 

8.  MODIFY, , ,NEWFET 

9.  FET, TEMP, SCRATCH 1 ,256 

10.  OPEN, 3 

11.  EQUIP, 01 =60, 02=61 , 1 0=MT 

12.  LOAD, 10 

13.  RUN 

14.  data 

15.  end  of  file 


These  control  cards  allocate  the  disk  file  OVERLAYS  (owner 
CECIL)  to  logical  unit  25  and  disk  file  SCRATCIU  (owner  TEMP)  to 
logical  unit  3.  Within  the  program,  logical  unit  25  is  used  to 
store  the  overlays  while  logical  unit  3  is  assumed  to  hold  ex¬ 
perimentally  collected  task  duration  and  interruption  data.  The 
control  cards  also  assume  that  the  program  is  stored  in  binary 
form  on  magnetic  tape. 


15 


_ _ 


s 


The  control  cards  to  store  the  program  on  tape  are: 

1 .  Sequence  card 

2.  Job  card 

3.  EQUIP, 10=MT 

4.  XFER, 10 
binary  cards 

5.  CTO, END 

6.  End  of  file 

7.  End  of  file 


p.  A-2  (Table  1)  Insert: 


Subscript 

Variable  FORTRAN 

Interruption  Category 

1-  Intermittant  reference  ITY 

2-  Accidental  interruption 

3-  Personal  comfort 

4-  Other 


Maximum 

Maximum  Value 

Value  FORTRAN 


9  ITYMAX 


NOTE:  Interruptions  can  be  assigned  for  a  maximum  of  3  task  analyses  arid 
10  task  elements. 


p.  A-5  (Table  A-3)  Replace  ORO  (3)-  Not  used . 17 

with 

ORO  (S)--Output  reading  option  8 

If  equal  to  1  read  in  experiment 
data  and  complete  interruption 
trials  and  new  task  analysis ....  17 


16 


p.  A-10  (Table  A-8)  Add: 


Number  of  interruptions  to  be  considered  on 
this  task  element.  This  value  determines  the 
number  of  interrupt  data  to  be  read  in  (INTS  <_  4 
read  1  additional  card:  5  <_  INTS  <_  9  read  2  addi¬ 
tional  cards) 

Type  of  interruption,  up  to  four  on  a  card 


PROBI (I ,K,ITYP)  Probability  of  this  type  of  interrupt 

AITE ( I , K, ITYP )  Average  duration  of  interrupt 

ADI (I ,K, ITYP)  Standard  deviation  of  interruption 


p.  A-12  (Table  A-10)  Add: 

NDCI (I,K,ITY)  Number  of  interrupts  of  type  ITY  on  task  element  I 
for  analysis  K 

NDC (I , K)  Number  of  experimental  ,  erformance  times  collected 

for  task  element  I  of  task  analysis  K 

DCITOT(I,K, ITY)  Sum  of  performance  times  for  interrupts 

DCTOT(I,K)  Sum  of  performance  times  for  task  elements 


2,22,42,62 

3-8,23-28 

43-48,63-68, 

9-14,29-34 

49-54,69-74 

15-20,35-40 

55-60,75-80 


p-i'  --V 


p.  A-13  (Table  A-10)  Add  to  the  nondimensioned  variables: 


NINTS 

Total  number  of  interruptions 

recorded 

NOPTI 

Total  number  of  experimental 
recorded 

task  performance  times 

NCHARS 

Total  number  of  characters  in 
experimentally  performed 

messages  which  were 

TMIN 

Duration  of  interrupt 

A-14 

(Table  A-ll)  Add: 

COMPU  Computation  Computes  interruption  and  new  task  analysis 

data. 


p.  B-2  Before  last  paragraph  add: 

The  experimenter  is  also  given  the  capability  to  indicate  the  interrup¬ 
tion  of  subject  performance  after  the  subject  has  begun  task  performance. 
Interruption  codes  of  1  through  9  are  used  to  classify  the  interruptions  and 
the  experimenter  can  enter  an  additional  20  character  description  of  each  in¬ 
terruption  . 
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p.  B-3  Before  supporting  subroutines  add: 


The  experimenter  indicates  task  performance  start  time  and  all  in¬ 
terruptions  in  this  mode  of  operation.  The  completion  of  a  task  is  indi¬ 
cated  when  the  subject  depresses  the  SEND  key  after  entering  the  TOS  data. 

In  either  case,  of  subject  performance  the  model  will  write  to  a  disk 
file  interruption  and  task  performance  data  to  be  processed  by  an  analysis 
program  in  the  foreground  mode.  If  this  option  is  selected,  incorrect  sub¬ 
ject  responses  are  not  displayed  to  the  experimenter. 


p.  B-4  Add: 

0R0(8)  -  Option  to  write  interrupt  and  task  performance  data  to  a 
disk  file  for  later  processing.  (Set  to  1  to  exercise 
option) 

INTRY  0-10)  -  Array  used  to  write  interruption  data  and  task  per¬ 
formance  data  to  disk  for  later  processing. 


p.  B-6  Add: 


NINTK= 

NOPTI- 

ISCI= 


ISCRNE= 

IEXM-” 


KT- 

IDIJR= 


Total  interrupt  time  (in  seconds)  for  a  task  performance 
Total  number  of  task  elements  by  subjects 

Mode  indicator  for  CRT  management  routine  (WDISP)  O:  WDISP 
WAITS  FOR  SEND  KEY  FROM  ONE  CRT.  1 :  WDISP  WAITS  FOR  SEND 
KEY  FROM  EITHER  EXPERIMENTER  OR  SUBJECTS  CRT.  2:  WDISP 
DOES  NOT  WAIT  FOR  SEND  KEY  WDISP  SETS  ISCI  TO  CRT  NUMBER 
FROM  WHICH  SEND  KEY  WAS  RECEIVED 
IDENTIFIER  OF  EXPERIMENTER  CRT 

Mode  indicator  for  experimenter  O:  No  interrupt  is  outstanding 
1 :  Interrupt  is  outstanding 

Interrupt  code 

Duration  of  current  interrupt 
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7 

p.  B~7  Add  alter  card  3  (9  Open, 2): 

7 

g  PET , TEMP , SCRATCH  1  ,  ,  , 

1  MODIFY, , ,NEWFET 

1  FET , TEMP , SCRATCH  1,256 
9 

7  OPEN, 3 
9 


p.  B-13  Change  16-18  3x  not  used  to 


16  IX  Not  used 

17  II  ORO(8) — if  1  write  interruption  and  task  performance 

on  disk 

18  IX  not  used 


p- 

B-22 

Add 

p. 

B-22 

Add 

If  the  option  to  write  interruption  and  task  performance  data  to 
disk  is  selected,  then  that  data  will  be  entered  onto  the  disk  reserved  for 
the  interactive  model.  This  same  disc  must  be  mounted  when  the  batch  model 
is  executed  to  process  the  recorded  data. 


20 


•  t*'  «»**-•*  ***** 


APPENDIX  B 


Revised  Flow  Charts  for  MAN  MODEL 


The  flow  charts  pr  esented  in  Appendix  B  represent  modifications 
to  prior  flow  charts  as  a  consequence  of  the  revisions  described  in  the 
body  of  this  report. 
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Figure  B-l.  Main  sequence  logic  flow  for  TOS  model  (Routine  Sips) 


1 


Operator  processing  subroutine  (Routine  Proc) 


Figure  B -5.  Computational  subroutine  (COMPU) 


DISK 


WRITE  INTERRUPT 
RECORD  IF 
ORO  (8)  =  1 


DISPLAY  TO  EXPERIMENTER:  EXPERIMENTER  ENTERS 

PERFORMANCE  TIME  =_  SECONDS  NUMBER  OF  ERRORS 

CONTINUED  FROM  ENTER  NUMBER  ERRORS-'*  OBSERVED  AND  S  OR  F 

PREVIOUS  PAGE  SUCCESS/FAILURE  (S/F)— *  "  FOR  SUCCESS/FAI LURE 

DEPRESS  SEND  TO  CONTINUE  INDICATOR 


